Skip to content

fix json format use llm#1275

Merged
iceljc merged 5 commits intoSciSharp:masterfrom
yileicn:master
Jan 15, 2026
Merged

fix json format use llm#1275
iceljc merged 5 commits intoSciSharp:masterfrom
yileicn:master

Conversation

@yileicn
Copy link
Member

@yileicn yileicn commented Jan 14, 2026 •
edited by qodo-code-review bot
Loading

PR Type

Enhancement

Description

  • Add JSON repair service using LLM for malformed JSON

  • Implement IJsonRepairService interface with repair methods

  • Create JsonRepairPlugin for dependency injection registration

  • Add Liquid template for JSON repair prompt generation

Diagram Walkthrough

flowchart LR
  A["Malformed JSON"] -->|RepairAndDeserialize| B["JsonRepairService"]
  B -->|Repair| C["LLM Agent"]
  C -->|json_repair.liquid| D["Repaired JSON"]
  D -->|Deserialize| E["Target Type T"]
  B -->|Direct Deserialization| F["Success/Default"]
Loading

File Walkthrough
Relevant files
Enhancement
IJsonRepairService.cs
JSON repair service interface definition                                 

src/Infrastructure/BotSharp.Abstraction/Utilities/IJsonRepairService.cs

  • Define IJsonRepairService interface with two methods
  • RepairAndDeserialize method for repair and deserialization
  • Repair method for JSON string repair only
 +23/-0   
JsonRepairPlugin.cs
Plugin registration for JSON repair service                           

src/Infrastructure/BotSharp.Core/JsonRepair/JsonRepairPlugin.cs

  • Create JsonRepairPlugin implementing IBotSharpPlugin
  • Register JsonRepairService in dependency injection
  • Provide plugin metadata with ID, name, and description
 +19/-0   
JsonRepairService.cs
JSON repair service implementation with LLM                           

src/Infrastructure/BotSharp.Core/JsonRepair/JsonRepairService.cs

  • Implement IJsonRepairService with LLM-based repair logic
  • RepairAndDeserialize attempts direct deserialization first
  • Repair method uses router agent and Liquid template
  • Handle exceptions and log repair results
 +99/-0   
json_repair.liquid
Liquid template for JSON repair prompt                                     

src/Infrastructure/BotSharp.Core/data/agents/01fcc3e5-9af7-49e6-ad7a-a760bd12dc4a/templates/json_repair.liquid

  • Create Liquid template for JSON repair prompt
  • Template instructs LLM to fix malformed JSON
  • Output valid JSON only from LLM response
 +3/-0     
Configuration changes
BotSharp.Core.csproj
Project file updates for template resources                           

src/Infrastructure/BotSharp.Core/BotSharp.Core.csproj

  • Add None Remove entry for json_repair.liquid template
  • Add ItemGroup with CopyToOutputDirectory for normalize_json.liquid
  • Update project file to include new template resources
 +7/-0     

@qodo-code-review
Copy link

qodo-code-review bot commented Jan 14, 2026 •
edited
Loading

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🔴
Sensitive data logging

Description: The code logs the full LLM output (response.Content) for “JSON repair result”, which may
include sensitive user-provided data from malformedJson and therefore can cause sensitive
information exposure in application logs.
JsonRepairService.cs [95-96]

Referred Code 
_logger.LogInformation($"JSON repair result: {response.Content}");
return response.Content;
⚪
Untrusted LLM output

Description: Untrusted malformedJson is sent to an LLM and the unvalidated response (response.Content)
is returned and later deserialized, creating a realistic risk of
prompt-injection/manipulated output (e.g., returning unexpected structure or very large
payload) that could lead to downstream misuse or resource exhaustion unless strict
validation/size limits are enforced.
JsonRepairService.cs [68-97]

Referred Code 
var render = _services.GetRequiredService<ITemplateRender>();
var prompt = render.Render(template, new Dictionary<string, object>
{
    { "input", malformedJson }
});

var completion = CompletionProvider.GetChatCompletion(_services,
    provider: router?.LlmConfig?.Provider,
    model: router?.LlmConfig?.Model);

var agent = new Agent
{
    Id = Guid.Empty.ToString(),
    Name = "JsonRepair",
    Instruction = "You are a JSON repair expert."
};

var dialogs = new List<RoleDialogModel>
{
    new RoleDialogModel(AgentRole.User, prompt)
    {


 ... (clipped 9 lines)
Ticket Compliance
⚪
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
⚪
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

🔴
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Swallowed exception: The code catches and ignores the initial deserialization exception without logging/context
and performs no explicit null/empty validation on malformedJson, reducing debuggability
and edge-case safety.

Referred Code 
public async Task<T?> RepairAndDeserialize<T>(string malformedJson)
{

    try
    {
        // First try direct deserialization
        return malformedJson.JsonContent<T>();
    }
    catch
    {
        // Continue to repair
    }

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status:
Sensitive data logged: The service logs the full LLM-produced repaired JSON (response.Content), which may contain
sensitive user-provided data, violating secure logging requirements.

Referred Code 
var response = await completion.GetChatCompletions(agent, dialogs);

_logger.LogInformation($"JSON repair result: {response.Content}");
return response.Content;

Learn more about managing compliance generic rules or creating your own custom rules

⚪
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
Missing audit context: The new JSON repair flow emits logs without user identity/correlation context, making it
hard to reconstruct who triggered the repair and its outcome if this action is considered
audit-critical.

Referred Code 
        _logger.LogError(ex, "Failed to repair and deserialize JSON");
        return default;
    }
}

public async Task<string> Repair(string malformedJson)
{
    var agentService = _services.GetRequiredService<IAgentService>();
    var router = await agentService.GetAgent(ROUTER_AGENT_ID);

    var template = router.Templates?.FirstOrDefault(x => x.Name == TEMPLATE_NAME)?.Content;
    if (string.IsNullOrEmpty(template))
    {
        _logger.LogWarning($"Template '{TEMPLATE_NAME}' not found in agent '{ROUTER_AGENT_ID}'");
        return malformedJson;
    }

    var render = _services.GetRequiredService<ITemplateRender>();
    var prompt = render.Render(template, new Dictionary<string, object>
    {
        { "input", malformedJson }


 ... (clipped 25 lines)

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status:
Unvalidated LLM input: The method forwards malformedJson directly into an LLM prompt without visible
validation/size limits/sanitization, which may allow prompt-injection or resource
exhaustion depending on how this input is sourced.

Referred Code 
public async Task<string> Repair(string malformedJson)
{
    var agentService = _services.GetRequiredService<IAgentService>();
    var router = await agentService.GetAgent(ROUTER_AGENT_ID);

    var template = router.Templates?.FirstOrDefault(x => x.Name == TEMPLATE_NAME)?.Content;
    if (string.IsNullOrEmpty(template))
    {
        _logger.LogWarning($"Template '{TEMPLATE_NAME}' not found in agent '{ROUTER_AGENT_ID}'");
        return malformedJson;
    }

    var render = _services.GetRequiredService<ITemplateRender>();
    var prompt = render.Render(template, new Dictionary<string, object>
    {
        { "input", malformedJson }
    });

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link

qodo-code-review bot commented Jan 14, 2026 •
edited
Loading

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
High-level
Consider a simpler, non-LLM alternative

It is suggested to replace the LLM-based JSON repair mechanism with a dedicated,
deterministic JSON repair library or a rule-based parser. This would improve
performance, reduce cost, and increase reliability by avoiding the overhead and
non-determinism of an LLM call.

Examples:

src/Infrastructure/BotSharp.Core/JsonRepair/JsonRepairService.cs [56-97] 
    public async Task<string> Repair(string malformedJson)
    {
        var agentService = _services.GetRequiredService<IAgentService>();
        var router = await agentService.GetAgent(ROUTER_AGENT_ID);
        
        var template = router.Templates?.FirstOrDefault(x => x.Name == TEMPLATE_NAME)?.Content;
        if (string.IsNullOrEmpty(template))
        {
            _logger.LogWarning($"Template '{TEMPLATE_NAME}' not found in agent '{ROUTER_AGENT_ID}'");
            return malformedJson;

 ... (clipped 32 lines)

Solution Walkthrough:

Before:

public class JsonRepairService : IJsonRepairService
{
    public async Task<string> Repair(string malformedJson)
    {
        // 1. Get an agent
        var router = await agentService.GetAgent(ROUTER_AGENT_ID);
        
        // 2. Render a prompt for the LLM
        var prompt = render.Render(template, new { input = malformedJson });
        
        // 3. Get a chat completion provider
        var completion = CompletionProvider.GetChatCompletion(_services, ...);
        
        // 4. Call the LLM to get the repaired JSON
        var response = await completion.GetChatCompletions(agent, dialogs);
        return response.Content;
    }
}

After:

// Using a hypothetical deterministic JSON repair library
public class JsonRepairService : IJsonRepairService
{
    public Task<string> Repair(string malformedJson)
    {
        try
        {
            // Use a deterministic, local library to fix common JSON errors.
            // This is faster, cheaper, and more reliable than an LLM call.
            string repairedJson = SomeJsonRepairLibrary.Fix(malformedJson);
            return Task.FromResult(repairedJson);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to repair JSON deterministically.");
            return Task.FromResult(malformedJson); // Or re-throw
        }
    }
}
Suggestion importance[1-10]: 9

__

Why: This suggestion correctly identifies significant performance, cost, and reliability issues with using an LLM for JSON repair and proposes a more standard, efficient, and deterministic architectural alternative.

High
Possible issue
✅ Copy JSON repair template
Suggestion Impact:The commit updated the .csproj to copy json_repair.liquid to the output directory. However, instead of adding a new entry alongside normalize_json.liquid, it replaced the normalize_json.liquid copy entry with json_repair.liquid.

code diff:

   <ItemGroup>
-    <None Update="data\agents\01fcc3e5-9af7-49e6-ad7a-a760bd12dc4a\templates\normalize_json.liquid">
+    <None Update="data\agents\01fcc3e5-9af7-49e6-ad7a-a760bd12dc4a\templates\json_repair.liquid">
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
     </None>
   </ItemGroup>

In the .csproj file, add a entry for json_repair.liquid with to
ensure the template file is copied to the build output and available at runtime.

src/Infrastructure/BotSharp.Core/BotSharp.Core.csproj [295-299]

 <ItemGroup>
   <None Update="data\agents\01fcc3e5-9af7-49e6-ad7a-a760bd12dc4a\templates\normalize_json.liquid">
     <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
   </None>
+  <None Update="data\agents\01fcc3e5-9af7-49e6-ad7a-a760bd12dc4a\templates\json_repair.liquid">
+    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+  </None>
 </ItemGroup>

[Suggestion processed]

Suggestion importance[1-10]: 8

__

Why: This suggestion correctly identifies a critical missing configuration for the new json_repair.liquid template, which is essential for the JsonRepairService to function correctly at runtime.

Medium
Learned
best practice
✅ Add null checks and fallback
Suggestion Impact:The commit increased defensive behavior by wrapping the LLM repair path in a try/catch, using null-conditional access on router LLM config, and falling back to returning the original JSON on exceptions. However, it did not add the explicit router-not-found warning/early return or the explicit response.Content null/empty checks and warning logs as suggested.

code diff:

-    public async Task<string> Repair(string malformedJson)
+    private async Task<string> RepairByLLM(string malformedJson)
     {
         var agentService = _services.GetRequiredService<IAgentService>();
         var router = await agentService.GetAgent(ROUTER_AGENT_ID);
@@ -71,18 +77,20 @@
             { "input", malformedJson }
         });
 
-        var completion = CompletionProvider.GetChatCompletion(_services,
+        try
+        {
+            var completion = CompletionProvider.GetChatCompletion(_services,
             provider: router?.LlmConfig?.Provider,
             model: router?.LlmConfig?.Model);
 
-        var agent = new Agent
-        {
-            Id = Guid.Empty.ToString(),
-            Name = "JsonRepair",
-            Instruction = "You are a JSON repair expert."
-        };
+            var agent = new Agent
+            {
+                Id = Guid.Empty.ToString(),
+                Name = "JsonRepair",
+                Instruction = "You are a JSON repair expert."
+            };
 
-        var dialogs = new List<RoleDialogModel>
+            var dialogs = new List<RoleDialogModel>
         {
             new RoleDialogModel(AgentRole.User, prompt)
             {
@@ -90,9 +98,15 @@
             }
         };
 
-        var response = await completion.GetChatCompletions(agent, dialogs);
+            var response = await completion.GetChatCompletions(agent, dialogs);
 
-        _logger.LogInformation($"JSON repair result: {response.Content}");
-        return response.Content;
+            _logger.LogInformation($"JSON repair result: {response.Content}");
+            return response.Content.CleanJsonStr();
+        }
+        catch (Exception ex)
+        {
+            _logger.LogError(ex, "Failed to repair and deserialize JSON");
+            return malformedJson;
+        }

Guard against router or response.Content being null (e.g., agent not found,
provider errors) and fall back safely to the original JSON while logging the
reason.

src/Infrastructure/BotSharp.Core/JsonRepair/JsonRepairService.cs [58-96]

 var agentService = _services.GetRequiredService<IAgentService>();
 var router = await agentService.GetAgent(ROUTER_AGENT_ID);
+if (router == null)
+{
+    _logger.LogWarning("Agent '{AgentId}' not found; returning original JSON.", ROUTER_AGENT_ID);
+    return malformedJson;
+}
 
 var template = router.Templates?.FirstOrDefault(x => x.Name == TEMPLATE_NAME)?.Content;
 if (string.IsNullOrEmpty(template))
 {
-    _logger.LogWarning($"Template '{TEMPLATE_NAME}' not found in agent '{ROUTER_AGENT_ID}'");
+    _logger.LogWarning("Template '{TemplateName}' not found in agent '{AgentId}'; returning original JSON.", TEMPLATE_NAME, ROUTER_AGENT_ID);
     return malformedJson;
 }
 ...
 var response = await completion.GetChatCompletions(agent, dialogs);
 
-_logger.LogInformation($"JSON repair result: {response.Content}");
-return response.Content;
+var content = response?.Content;
+if (string.IsNullOrWhiteSpace(content))
+{
+    _logger.LogWarning("JSON repair returned empty content; returning original JSON.");
+    return malformedJson;
+}
 
+_logger.LogInformation("JSON repair result: {Json}", content);
+return content;
+

[To ensure code accuracy, apply this suggestion manually]

Suggestion importance[1-10]: 6

__

Why:
Relevant best practice - Improve defensive coding with precise null and state checks to prevent crashes and incorrect behavior.

Low
✅ Log exceptions instead of swallowing
Suggestion Impact:The commit refactored the flow so direct deserialization is no longer wrapped in a swallow-all catch; instead, it validates JSON separately and then deserializes without catching (so failures aren't silently swallowed). Additionally, it added a try/catch with LogError around the LLM-based repair call. However, it did not implement the specific suggested LogDebug in the "direct deserialization failed" catch (that catch was removed rather than updated).

code diff:

-        var completion = CompletionProvider.GetChatCompletion(_services,
+        try
+        {
+            var completion = CompletionProvider.GetChatCompletion(_services,
             provider: router?.LlmConfig?.Provider,
             model: router?.LlmConfig?.Model);
 
-        var agent = new Agent
-        {
-            Id = Guid.Empty.ToString(),
-            Name = "JsonRepair",
-            Instruction = "You are a JSON repair expert."
-        };
+            var agent = new Agent
+            {
+                Id = Guid.Empty.ToString(),
+                Name = "JsonRepair",
+                Instruction = "You are a JSON repair expert."
+            };
 
-        var dialogs = new List<RoleDialogModel>
+            var dialogs = new List<RoleDialogModel>
         {
             new RoleDialogModel(AgentRole.User, prompt)
             {
@@ -90,9 +98,15 @@
             }
         };
 
-        var response = await completion.GetChatCompletions(agent, dialogs);
+            var response = await completion.GetChatCompletions(agent, dialogs);
 
-        _logger.LogInformation($"JSON repair result: {response.Content}");
-        return response.Content;
+            _logger.LogInformation($"JSON repair result: {response.Content}");
+            return response.Content.CleanJsonStr();
+        }
+        catch (Exception ex)
+        {
+            _logger.LogError(ex, "Failed to repair and deserialize JSON");
+            return malformedJson;
+        }

Avoid swallowing exceptions; at minimum log the exception so callers/operators
can understand why parsing failed before repair is attempted.

src/Infrastructure/BotSharp.Core/JsonRepair/JsonRepairService.cs [33-41]

 try
 {
     // First try direct deserialization
     return malformedJson.JsonContent<T>();
 }
-catch
+catch (Exception ex)
 {
+    _logger.LogDebug(ex, "Direct JSON deserialization failed; attempting LLM repair.");
     // Continue to repair
 }

[Suggestion processed]

Suggestion importance[1-10]: 5

__

Why:
Relevant best practice - Preserve error details by capturing exception messages (and other error output) instead of swallowing exceptions and returning generic defaults.

Low
  • Update

iceljc
iceljc reviewed Jan 15, 2026
View reviewed changes
@yileicn
Co-authored-by: adenchen123 <cykb0903@gmail.com>
f5f0381 
Co-authored-by: Aden <128471249+adenchen123@users.noreply.github.com>
Co-authored-by: Jicheng Lu <franklujc@gmail.com>
Co-authored-by: iceljc <46039045+iceljc@users.noreply.github.com>
Co-authored-by: Jack <jjy_8888@163.com>
Co-authored-by: jackjiang-sms <jianyong.jiang@starlight-sms.com>
optimize ResponseFormat
@iceljc
Copy link
Collaborator

iceljc commented Jan 15, 2026

The current changes cannot be compiled.

@yileicn
Copy link
Member Author

yileicn commented Jan 15, 2026

The current changes cannot be compiled.

Fixed

@iceljc iceljc merged commit 022a175 into SciSharp:master Jan 15, 2026
4 checks passed
@JackJiang1234
Copy link
Contributor

JackJiang1234 commented Jan 15, 2026

reviewed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iceljc iceljc iceljc left review comments

@Oceania2018 Oceania2018 Oceania2018 approved these changes

Assignees

@iceljc iceljc

Labels

Review effort 3/5

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yileicn @iceljc @JackJiang1234 @Oceania2018